Report an aggregate token-volume-weighted compression percentage - #52
Conversation
… percentage avg_compression_pct is Headroom's own unweighted average of each request's individual compression ratio, so a run with one huge, highly-compressible request and many small, barely-compressible ones understates how much of the actual token volume was saved. Add headroom_total_percent_saved, computed as total_tokens_removed / total_tokens_before across every proxied request this run, and surface it alongside the existing average in the Step Summary, the PR comment, and a new output.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
…n note The comment documenting which /stats fields were confirmed empirically against a live response (and must be re-checked on every future pin bump) named only total_tokens_removed/avg_compression_pct. total_tokens_before is exactly as undocumented and exactly as load-bearing, since headroom_total_percent_saved depends on it, so name it there too.
Superseded by re-review on the latest commit.
There was a problem hiding this comment.
Re-review at abfd46a. The only finding from my prior review (Nit: the "confirmed empirically" note at action.yml:1706 didn't name total_tokens_before alongside total_tokens_removed/avg_compression_pct) is fixed correctly in this commit — the field is now listed there, matching the suggestion I made, and the thread is resolved. No other changes since the prior revision, and I found nothing new to raise.
I dismissed and cleared my prior stale APPROVED review (submitted against the earlier commit) per the re-review convention, since a new commit landed after it.
Nothing blocking. Approving.
Disclosure: as on the prior revision, gh pr list/gh issue list are blocked by a tool-approval restriction in this run, so I still could not check for related open PRs or a tracking issue.
|
🗜️ Headroom compressed this run's context: proxied 12 request(s), saving 44004 tokens, 2.8% of all tokens sent this run (3.8% average per-request compression). |
|
🎉 This PR is included in version 1.9.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
1 participant
Headroom's own avg_compression_pct is an unweighted average of each request's individual compression ratio. Confirmed via a local test with mixed request sizes that this can diverge substantially from the real aggregate token-volume savings: a 74.5% average alongside a 97.9% aggregate for the same run, because one huge highly-compressible request pulls far more actual token weight than several small moderately-compressible ones, even though it counts as only one data point in the average.
Adds
headroom_total_percent_saved, computed from the proxy's owntotal_tokens_removed/total_tokens_beforefields, and surfaces it alongside the existing average in the Step Summary, the PR comment, and as a new output.Test plan
run:script viayaml.safe_load()and ran it against a live local Headroom proxy (pip venv) with a mix of request sizes, confirming the aggregate and average figures genuinely diverge and both compute correctlyworkflow_callwrapper: no new findings at the two modified stepsnpm run lint/typecheck/format:checkall pass